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(54) HEAD MOUNT DISPLAY DEVICE 
(57)Abstract: 

PROBLEM TO BE SOLVED: To designate a desired 
position in a visual field by overlapping an image like an 
icon as a virtual image on a real visual field and moving 
the icon image through a sight line in the case that the 
mixed reality can be reproduced. 

SOLUTION: An output section 5 displays an icon image 
at a sight line position detected by a sight line detection 
section 8 through the entry from an operation entry 
section 6. The operator views an icon image overlapped 
on a real visual field seen through a window. The 
operator brings the sight line to a desired position in the 
visual field and decides the overlapped image through an 
instruction from the operation entry section 6. For 
example, when the operator observes a sentence 
actually in front of its eyes and designates an area in the 
sentence. An image processing section 7 discriminates a 
sentence or a word in the designated area based on 
image information received from a CCD camera section 
3, translates it into an optional language and the image 
output section 5 displays the data. 
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* NOTICES* 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2, **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim l] The head mount display device which can pile up and observe a fictitious picture within an 
operator's field of view characterized by providing the following The CCD camera section which picturizes 
an operator's field of view The visual-axis detecting element which detects an operator's direction of a 
visual axis The operation input section for indicating the fictitious picture by move on a display screen 
according to an operator's direction of a visual axis, and specifying arbitrary fields with the position of a 
fictitious picture The image-processing section which processes the image information of the field 
specified among the image information picturized in the CCD camera section 

[Claim 2] The head mount display device characterized by preparing a means to change the image 
processing data based on visual-axis position data into a desired gestalt in the image-processing section 
in invention of a claim 1. . 

[Claim 3] The head moimt display device characterized by preparing a dictionary function in a means to 
change in invention of a claim 2. 

[Claim 4] The head mount display device characterized by preparing the function which can communicate 
with a personal computer for a means to change in invention of a claim 2. 



[Translation done.] 



* NOTICES* 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 

precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings; any words are not transliated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[The technical field to which invention belongs] the fictitious body image by which this invention was 
displayed on the display, and the image with which the operator went into the field of view actually 
piling up **** it is related with the head mount display device which can do things 
[0002] 

[Description of the Prior Art] A head is equipped like goggles and the head mount display device (HMD is 
called hereafter) reproducing the "mixed reality" (reality and fictitious composite) which can see 
simultaneously what is actually being seen, and a fictitious body is realized. This example of HMD is 
explained with a drawing. Drawing 2 is the external view of HMD, drawing 3 shows the outline of a 
internal structure and drawing 4 is the block diagram of the system which constitutes HMD. This HMDl 
is what unified a window part 2, the CCD camera section 3, a one way mirror 4, and the picture output 
section 5, and when it equips like drawing 2 , from a window part 2, what actually exists at hand can be 
seen. Moreover, although it is in an operator's field of view from the CCD camera section 3, an image is 
incorporated as image data. HMDl is further equipped with the operation input section 6 and the 
image-processing section 7, the output of the CCD camera section 3 and the operation input section 6 is 
connected to the image-processing section 7, and the output of the image -processing section 7 is connected 
to the picture output section 5 which consists of a small liquid crystal display. 

[0003] The object which an operator has at hand actually can see the light A which enters from the 
exterior through a window part 2 by the eye through a one-way mirror 4. Moreover, the image 
information of a fictitious object is memorized by the operation input section 6, and the image of the 
specified fictitious object is projected on the picture output section 5 on either side. The picture B reflects 
a one way mirror 4, and can observe it now by both eyes. The image C which compounded the object 
which exists actually, and the fictitious object by this composition can be seen by both eyes. Moreover, it is 
possible to show a fictitious picture in three dimensions by giving binocular parallax to the fictitious 
picture which is in sight by the eye on either side. 

[0004] Here, suppose that a desk is at hand actually. When the setting input of what is placed on a desk is 
carried out fi-om the operation input section 6, the inputted fictitious object seems to exist simultaneously 
with an actual object. For example, although an actual desk will move to the direction of the left in a field 
of view if the pen stand which is not in fact is inputted from the operation input section, it seems that a 
pen stand is on a desk when it specifies that it displays on the predetermined position of the picture 
output section 5, and a neck is shaken at the right in this state (if a visual axis is shaken at the right) In 
connection with this, a fictitious pen stand also moves the position on a desk to the direction of the left in 
a field of view, with the original position maintained. 

[0005] Although an actual desk wiU move to the direction of the left in a field of view (aperture) if a neck 
is shaken at the right (if a visual axis is shaken at the right) In order that the picture caught in the CCD 
camera section 3 may also move to the direction of the left, carry out the image processing of the image 
from this CCD camera section 3, calculate the move direction and movement magnitude, and the 
fictitious move direction of a picture and movement magnitude are drawn. If the position on a screen is 
amended, a fictitious picture (pen stand) can reproduce a state [ having stood it still in the original 
position of a desk ]. 

[0006] As mentioned above, although there was a system which can pile up conventionally what is 
actually visible, and a fictitious object, and can be seen simultaneously, there was nothing that specifies a 
position to be what is actually visible by the visual-axis input, and processes the information on the 
specified field. 
[0007] 

[Problem(s) to be Solved by the Invention] this invention piles up an image image like an icon as a 
fictitious image with an actual field of view, moves this icon image and enables it to specify the position of 



the request in a field of view by the visual axis in HMD which can reahze a mixed reality. 
[0008] 

[Means for Solving the Problem] In the head mount display device which invention of a claim 1 can lay a 

fictitious pictxure on top of an operator's field of view, and can be observed The CCD camera section which 

picturizes an operator's field of view, and the visual-axis detecting element which detects an operator's 

direction of a visual axis, The operation input section for indicating the fictitious picture by move on a 

display screen according to an operator's direction of a visual axis, and specifying arbitrary fields with the 

position of a fictitious picture, It has the image-processing section which processes the image information 

of the field specified among the image information picturized in the CCD camera section. 

[0009] Invention of a claim 2 prepares a means to change the image processing data based on visual-axis 

position data into a desired gestalt in the image-processing section in invention of a claim 1. 

[0010] Invention of a claim 3 prepares a dictionary function in a means to change, in invention of a claim 

2. 

[OOll] Invention of a claim 4 prepares the function which can communicate with a personal computer for 

a means to change in invention of a claim 2. 

[0012] 

[Embodiments of the Invention] The composition of this invention is shown in drawing 1 . As compared 
with the composition of the conventional example this invention was indicated to be to drawing 4 , it has 
the visual-axis detecting element 8 further, and the output is connected to the image-processing section 7. 
If an operator moves a visual axis to the object in a desired actual field of view, what detects the eye 
movement of the operator at that time etc., detects the direction of a visual axis, applies infrared 
radiation to an eye, for example, detects the direction of the reflected light by the sensor (CCD) for 
visual-axis detection is put in practical use, and it should just be used for the visual-axis detecting 
element 8. An image image like an icon is displayed on the picture output section 5 by the input from the 
operation input section 6. It is made for the display position on the display screen of the picture output 
section 5 of an icon to turn into a visual-axis position detected by the visual-axis detecting element 8. An 
icon image laps with the actual field of view which is visible from a window part, and it is observed by an 
operator's eyes. If a visual axis is moved, an icon image will also move in a screen top according to it. The 
operator has in the part of the request of a visual axis in a field of view, and decides with the directions 
from the operation input section 6. For example, the visual-axis detecting element 8 and the operation 
input section 6 are used, the text which exists at hand actually is observed, and the field in the text is 
specified. The text or word in the field specified by this is deciphered in the image-processing section 7 
from the image information into which it was inputted from the CCD camera section 3, it translates into 
arbitrary languages, and the data is displayed in the picture output section 5. In this case, display 
positions (for example, the upper right angle of a field of view, a lower left angle, etc.) can also be specified 
by the operation input section 6. 

[0013] In addition, although the method of presentation explained the example of the method which used 
the conventional small liquid crystal display, when a monochromatic display is also enough, the method of 
presentation which uses as the light source an LED array which was described in "a stereogram image 
synthesizer unit and its drive method" by Japanese Patent Application No. No, 311577 [ six to ] may be 
used [ a display may be not ****** but a character, and ] for it. [ finishing / people / these / application ] 
[0014] Moreover, it can change into the data which can transmit the thing of the field which chose the 
predetermined field using the above-mentioned method, and was chosen among the image information 
picturized in the CCD camera section 3 to a personal computer, and can also transmit to a personal 
computer. A picture or a character is sufficient as this data. This will have the function of the handy 
scanner which transmits the picture or characters of arbitrary parts, such as a magazine, a newspaper, 
and a photograph, to a personal computer as data, and the camera function to also download the picture 
of desired scenery, a person, etc. to a personal computer. 
[0015] 

[Effect of the Invention] In HMD reproducing a mixed reality, it can have a visual-axis detection means, 
selection specification of the object in an actual field of view can be carried out by the movement of an 
operator's visual axis, and the image information in a field of view can be processed by making a visual 
axis into an input means. 



[Translation done.] 



* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
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2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing l1 It is the system configuration view of this invention. 
[Drawing 21 It is the external view of a head mount display. 
[Drawing 31 It is the internal configuration view of a head mount display. 
[Drawing 41 It is the conventional mixed reality structure-of-a- system view. 
[Description of Notations] 

1 [ - The CCD camera section 4 / ■- A one-way mirror, 5 / The pictxu-e output section, 6 / The operation 
input section, 7 / The image -processing section, 8 / - Visual-axis detecting element. ] HMD, 2 -- A 
window part, 3 
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r tictt^cDteic^^^-iiifficoiTi-tj^it. -a-ff-4 
2 oti±W;^;> 7 Srffll^S r i J; t) r 

[0 0 2 7] m\K^-mmm\^i^^^x\±^ tf^^6(:i 
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TK^-^fifeL, •t=^-piS2{c;itb^^?r:-r-5, 071^ 

IIJi6«»JtC*;i^T:H. -f.V^J'-^^s/ b^^tj^^^^OT^- 
So 

[0028] ;iroJ;9fc, i5fe3D-& i « 5 7— gp^J-fciiftJ 

i-i.htzf)-^m.^W.X\,^^X.o r i: 

ii^x^h<r>x^ mx.\ii.im.^tz.\-i.t^'oxz.(D^m-^<r>m 
mm%. im^^n^ t^. ^(Dmt 

rid J; "9. ^(0*m^<eB#rs1(--^-x^(0#< (^tfffi 
5:Stt)Aix?):::i:;iSX-#5„ r(Oj5Effi-&(-;^ 
e-;*Srl9:»t, *^JcJ;oTt>1f«%St!9AtuS;ii:;5S 

[ 0 0 2 9 1 * fc, it^^^fcjES D -c (D^ - ^ -iSffitcr 

oSimc J; o T ^ -H® $ ixTi/ ^ 5 

3'-fe-i?dSrw^<7)A{-*f-r'5t(OTfcSir#{C{4, i 

fBcoJ: s'-lr— Sr ir^BJH±(-^^r;L 

ti.5rS± L-cms w i: ^{5;1S t hx^ So 
[0 0 3 0] ±fE<0J:9''i7'v?i5'/l'5 7-i^lSSr*^»- 

«T'ife I? f+tt S r i t^X^ So l^l21(c^-r«sj{cioi/^T 

fca5^>»->^7X-v, fSfB«iw»v^*-^-iii®2<^gag 

Lxv=^/U5 L-TTl^So Sfc. rcD^ = <? 

-iBffi 2 (^MffiiJicfi^ 1 !S:t-tr >'f- 3 i:^ 2 tgl:-ir 

•§-Sr^=^-iii®2{cm;'3L.Ti^S„ -©^-i? 
-Hffi2«±T*(Sj{c:-5riti{cS^^U, -^rWlEfiS^afcJ; 
50 oTfiJffl^(OW<^ft^f-. fi|ffl#WSIf^(cJ;oT, fcS 
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[0 0 3 11 ±ffi«J;9''.Cgafc*3V^T. i|#(c|illWtf 

-Ct, «sj;t«iii»*a!ieSPT-<4@4 (b) cDIEBlp] # (DM 
^^'t'-d^ic LT. 1^0 (a) (CTF-TJ; 9t-*i|ffl#ol4*!) 
£{|iJ<oi}!)i«(c:^;i. $)?>v^ttP[gl (b) (cjj^-r J; 9 fcHJ 
4 (c) lc:*-rJ;5{-Wffl#«^*:&ftiJ»l*«K{-^xT 

[0 0 3 21 -^rWt #<39:ftafi4ftE(-fHSUT 

[§14 (d) (c^-f-J; 5^c®S^^otf^^i^:^=&*^^:L-T'^b 

[0 0 3 3] W-»^6cOx-^f^)5!EgC9-e(±. tiT 

r i: /iST-# €>(DT% H 5 (d^T^I" J; ■p (ciijSWT 

[003 41 HI 5 (CliT-r =^>=S:^tt^(Dm^m^^ L 
, [0 0 3 51 ±E^«tfffil4. M^(J^Jffl*<^SiBE4'lc 



[0 0 3 6l iiiS±<DT'(' UTfi-tWte, 
Pii«i2! S It j]^ ^ X 5 «) <0 T -f 3 > ^ (i X T t ^ 5 o 

So 

[0 0 3 71 -tWftil, |gI**liMfC*5t>TI±'<'>'^— 4^ 

[0 0 3 81 ^fc. -f V^'-^ J- hS:fiJfflLfc7=-^i^ 
J; oT rtu^tT 9 r i t-et 

[0 0 3 91 |g|^^^;^Mt-*51'^XttM(' Txl/t-J COT 
7^nf®ffiS:^*-r'5:ii:;!)S-C'#5J:5tJ:tX*5fJ, fij 
JiSte^Tw wX'E'5wi:*5X't?.J; pfcuxv^So Mlc 

xii ^t3^<:$^. ^'s^muti/^x ! j rf^is, oo 

40 ?r{sx.Sr t/iS-r-tS^ B(rteirP«S-p< :x 

tX#5o 

[0 0 4 0] *^PJ«±fEO^J(73flll. Bi^fttStCjzoX 

SL^fc'ttX-j'i < &**s^<^)4 * Wiii^Sr^^1-5 r i: t 

"rtg/=eC»X\ fiJffi#;45ffifflL.(c< v^ilSDfcirttcd- 
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[0 04 11 

^ 5 ^ -BE 5 r Srit#m ^ ^ 5 

xv^i?/P5 7-igSt Lfc<©T\ «^-)rV1J-(Or-;5'Sr 

[0 0 4 21 ffjR]S2iC#,S^?g(w*Dl,^T(4. Sfr 

[0 0 4 31 W^^StC'^S^B^lc^si.^Ttt, HU 

[0 0 4 41 if*«4(vH?^5^?^(l< mrfe®^*!L 

[0 0 4 51 11*^5 {C^S^BJll. HfJlE!a«:-tr 

^tltiir t> OtfJia 3 Ji>:5nH^x- (c J; 9 f 
<D#^0 |S]t^(7)Bi«5:#6ri:^!|#mt 1-S M*^ 1 IE 

aSrtr o^t ^5T'# , .fil«#(OftK<D:ft^<DIE«t/cCB^ 

[0 0 4 61 *fcs f**^6(c^S^0J(i. mifEB^ia 

[0 0 4 71 4fc, f«*«7ic:«s^§^{i, mmmm!& 
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fc, ^=^5'-BBroS^OBm«-^#«':cv^SlS^t-#a 
[0 0 4 81 ilfc. ^*3S8{C#5^M«. Wistsi:* 

[0 0 4 91 %tz. m^^9\Z%:b^m\t. mW^"^ 
*3S7fB«gWr-'v'^/i^5 y-ma.t Ltz(DX\ ^<co%\\ 

[0 0 5 01 4fc. if*3ii oic-^s^B^tt. Mfeigl: 

«^T:-r S i X t , ffif iJ^ix v^^ 5 7 -^S i: -r S 
30 ri:/4ST-tSo 

[00 5 11 if 1 1 J-«S«W«:> MISlfx 
^(r>mm.ii., Mia*Jffl#<OBili:fiJffl#oe«^/^-efe 

Sr i:5r#mt-f SWj^^I OfSttWr-'i^;?/!-? 7-» 

[0 0 5 21 *fc/ff*«i 2(i#s^0J(4, tfrtetis: 

*iLagPT-{l#^$nfcflJffl#{cmLfctf^5rtll>3-t-S 

ffl#{c*f-rsif s »!ie<3(ca«-r s c t ^jsx-t s j: 9 

tc/^So 

[0 0 5 31 4fc. fS*^l 3{i«S^W«:, MIE*= 

mt-rsfi*^17l;^t»*«l 2c7)^^t=i^A— otcieig 

B^*!!iSILfcg^WB$i0^1,Sr ,J:;5SX-#. fiJfflttO^ 
7-i:1-Sri:;iSX*^So Sfc. r oStB-^iClSlt 

50 tc'fi^-f'/i-i y-iicnLx^mmm^m^i-?>mai. 
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G 0 6 T 1/00 3 4 0 



F I 

G 0 6 T 1/00 



f-rj-y (##) 



3 4 OA 



j^O < rtifiH 1 T @ 1 # 4 



F^— 5B057 AAOl BA02 CAOl CA08 CA12 
CA16 CBOl CB08 CB13 CB16 
CD14 CEOS 
5C054 AAOl CA04 CD03 CE04 CE16 
CH03 EAOl EA05 EHOl FA04 
FB03 FDOl FD07 FEU FE15 
FE18 FE23 HA36 



